Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

589-fixRunnerExamples #654

Closed
wants to merge 18 commits into from
Closed

589-fixRunnerExamples #654

wants to merge 18 commits into from

Conversation

TobiasNx
Copy link
Contributor

@TobiasNx TobiasNx commented Jan 28, 2025

This PR translates the existing workflow examples to fix and moves the old morph examples into the morph folder. See #589

I think I was able to translate all tests and adjust the workflows with regard to newer metafacture developments. The only test I was not able to translate was: metafacture-runner/src/main/dist/examples/count/metrics

Not sure if we delete one of metafacture-runner/src/main/dist/examples/gnd/crisscross-connections or metafacture-runner/src/main/dist/examples/gnd/references2 since they are identical.

@TobiasNx TobiasNx requested review from dr0i and blackwinter and removed request for dr0i February 3, 2025 16:33
@TobiasNx TobiasNx marked this pull request as ready for review February 3, 2025 16:34
@blackwinter
Copy link
Member

Not sure if we delete one of metafacture-runner/src/main/dist/examples/gnd/crisscross-connections or metafacture-runner/src/main/dist/examples/gnd/references2 since they are identical.

They're not identical. The extract.xml transformation differs as well as the generated output (crisscross-connections produces an empty output file).

@blackwinter
Copy link
Member

The only test I was not able to translate was: metafacture-runner/src/main/dist/examples/count/metrics

That's due to #651, right?

@TobiasNx
Copy link
Contributor Author

TobiasNx commented Feb 4, 2025

The only test I was not able to translate was: metafacture-runner/src/main/dist/examples/count/metrics

That's due to #651, right?

No morph is using a square collector, that I dont know how to translate it.

@blackwinter
Copy link
Member

Can we organize the examples a bit better? The top-level directories should be descriptive of the example's use case; there should be no generic morph (or fix) directory. All Metamorph Flux and transformation files should not include morph in the name, but in the extension instead (.morph.flux, .morph.xml [or maybe just .xml?]). Then the Metafix counterparts can be named accordingly (.fix.flux, .fix) and input files can be reused. No need to duplicate everything. And this would make it easier to compare both transformations. WDYT?

@TobiasNx
Copy link
Contributor Author

TobiasNx commented Feb 4, 2025

Not sure if we delete one of metafacture-runner/src/main/dist/examples/gnd/crisscross-connections or metafacture-runner/src/main/dist/examples/gnd/references2 since they are identical.

They're not identical. The extract.xml transformation differs as well as the generated output (crisscross-connections produces an empty output file).

I estimated that the morph transformation both had errors:
The gnd entity had no flushWith="record" option so that the examples did not create any output.

Additionally refrences2 used ; as delimiter when collecting sys, ent, etc. This is contradictory the fact of creating csv-lines delimited by ; as output:

<combine name=""
value="${count};${id};${nid};${satzart};${katlevel};${name};${sys};${ent};${syn}">
This would result in a broken csv file.

So the diffs of extract.xml are:

whitespaces and linesbreaks

a faulty delimiter

and filter elements that are not used in the further process.

$ diff '/home/tobias/mfExmplesOld/examples/gnd/references2/extract.xml' '/home/tobias/mfExmplesOld/examples/gnd/crisscross-connections/extract.xml'
198c198
<             <data source="@name" name="name"/>
---
>             <data source="@name" name="name" />
213c213
<                 <concat delimiter="; " name="sys">
---
>                 <concat delimiter=" | " name="sys">
224c224
<                 <concat delimiter="; " name="ent">
---
>                 <concat delimiter=" | " name="ent">
240d239
<         </entity>
241a241,265
>             <data name="filter1" source="004B.a">
>                 <blacklist>
>                     <entry name="gib" />
>                     <entry name="gio" />
>                     <entry name="giw" />
>                 </blacklist>
>             </data>
> 
>             <data source="[email protected]" name="filter2">
>                 <whitelist>
>                     <entry name="Ts1" />
>                     <entry name="Tg1" />
>                 </whitelist>
>             </data>
> 
>             <choose name="filter3">
>                 <data source="037G.c" />
>                 <data source="_id">
>                     <constant value="ok" />
>                 </data>
>                 <postprocess>
>                     <equals string="ok" />
>                 </postprocess>
>             </choose>
>         </entity>

output.xml is identical.

In my opinion this is a redundant example.

@blackwinter
Copy link
Member

In my opinion this is a redundant example.

Okay, fine. crisscross-connections was added in culturegraph/metafacture-runner@123a62e without any details given; references2 was added a week earlier in culturegraph/metafacture-runner@42d65f5.

TobiasNx added a commit that referenced this pull request Feb 4, 2025
Reuse all fix workflows from #654 and bring folders together as suggested by @blackwinter in #654 (comment)
TobiasNx added a commit that referenced this pull request Feb 5, 2025
Reuse all fix workflows from #654 and bring folders together as suggested by @blackwinter in #654 (comment) and in #662
@TobiasNx TobiasNx closed this Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants